feat(cli): implement runtime rule execution - #50
Merged
thecodedrift merged 12 commits intoJul 6, 2026
Merged
Conversation
Group 1 of runtime-rule-execution. Add the harness-side recognition of runtime rules: a directory under .taskless/runtime-rules/ holding kind: runtime capture *.yml plus a check.ts. - src/types/runtime-rule.ts mirrors the structural harness<->check contract (Finding, Match, CheckFunction, CaptureRule, metadata block) from the generator's @taskless/types; a delivered check imports nothing, so the contract is structural. - src/rules/runtime/discover.ts enumerates .taskless/runtime-rules/, parses each capture *.yml, confirms kind: runtime, and returns a typed RuntimeRule (capture rules with id/name/language/match + the check.ts path). .taskless/runtime-rule-tests/ is never enumerated. Refs TSKL-245 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Groups 2-3 of runtime-rule-execution. Implement the local harness that evaluates a runtime rule, plus the bundled tsx loader it runs check.ts under. - narrow.ts: run a rule's capture rules as ONE ast-grep scan (anchor --json=stream, broad --files-with-matches) and normalize matches to the contract shape (0-indexed -> 1-indexed, ruleId -> model name, captures from metaVariables). Uses a temp --config rules dir so multiple captures + full ast-grep config run in a single scan. - invoke.ts: run check.ts's default export (root, matches) via a pinned tsx resolved at runtime (no repo toolchain). An embedded ESM runner writes the returned Finding[] to an out-file; a throw, non-zero exit, or timeout is isolated to an error result. Default 10s bound, overridable. - harness.ts: narrow -> gate-on-matches -> invoke -> map Finding to CheckResult (source: taskless-runtime); process-per-check, sequential. - Add tsx to the CLI dependencies (externalized from the Vite bundle). Verified end-to-end against a temp-dir fixture: discovery, narrow (correct line normalization + rule attribution), tsx invocation, and finding mapping. Refs TSKL-245 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…d rules
Group 4 of runtime-rule-execution. Add the runtime reconcile-set: sign each
rule's check.ts (only) for reconciliation, select the rules the server blessed,
and materialize them for read-hash-execute.
- src/rules/runtime/run-set.ts: signRuntimeChecks (check.ts only — capture *.yml
are inert), reportRuntimeChecks -> { file, signature }, selectBlessedRuntimeRules
(content-join: a rule runs iff its check.ts signature is in the run set; the
rest are withheld/advisory), and materializeRuntimeRules (copy blessed rule
dirs into .taskless/.run/runtime-rules/ and re-discover so execution uses the
blessed bytes; .run/ stays gitignored).
- discover.ts: extract discoverRuntimeRulesIn(root) so materialized rules can be
re-discovered from .run/.
- narrow.ts: copy the original capture *.yml bytes into the temp config instead
of re-serializing the parsed object — a YAML round-trip can alter an exotic
ast-grep config.
Verified end-to-end: report only check.ts, bless on signature match, materialize
to .run/, execute the materialized copy; empty run set withholds the rule.
Refs TSKL-245
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ile gating Group 5 of runtime-rule-execution. Rewire `taskless check` around the two rule classes and complete the cutover to signing only untrusted code. - Static ast-grep rules under .taskless/rules/ always run (trusted, no network). - Runtime rules run only on a validated path: planRuntime resolves the mode from auth state + flags — authed reconcile runs blessed rules and withholds the rest (advisory); logged-out / --anonymous / no-remote / reconcile-unavailable skip runtime with a notice; --dangerously-run-scripts runs all runtime rules with no network behind a loud warning. - Add --dangerously-run-scripts and --timeout <seconds>; runtime findings merge into the same results and exit-code logic; --json gains an additive optional `skipped` array (schema updated), warnings/notices stay stderr-only. - Fix a Finding->CheckResult off-by-one: findings are 1-indexed, CheckResult.range is 0-indexed (display/json add 1). Cutover: remove the stacked-under static-reconcile gating — delete src/rules/run-set.ts and the now-obsolete test/reconcile-check.test.ts + test/run-set.test.ts (runtime-dispatch tests land in Group 7). Static rules are no longer signed or gated; only runtime check.ts is. Verified end-to-end via the built CLI: static-only-runs (runtime skipped + notice), --dangerously-run-scripts (both run), and --json (skipped array, warnings suppressed). Full CLI suite green (326 tests). Refs TSKL-245 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Groups 6-7 of runtime-rule-execution. Docs (Group 6): - check.txt (topic v2): static rules always run; runtime check.ts runs only when server-verified; the mode table, --dangerously-run-scripts, --timeout, and the --json skipped array. - ci.txt: unauthenticated CI runs static rules and skips runtime; the TASKLESS_TOKEN backstop is the enforcement point for runtime check.ts. Tests (Group 7): - runtime-harness.test.ts: discovery, gate-on-zero-matches (check never invoked), match normalization + Finding->CheckResult indexing, throwing-check isolation, timeout -> error finding. - runtime-check.test.ts: end-to-end dispatch via the built CLI with a mock reconcile server + git origin — authed-blessed, empty-run withheld, logged-out and --anonymous skip + report, reconcile-unavailable skips, dangerously-run- scripts runs offline; asserts static always runs and only check.ts is reported. Also fix a real timeout bug found by the harness test: tsx re-execs node as a grandchild, so spawn detached and SIGKILL the whole process group — otherwise a runaway check keeps running past the timeout. Full suite green (338 tests). Refs TSKL-245 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Implements local runtime rule execution for taskless check, adding a harness that runs runtime rules’ check.ts only when server-verified (or explicitly forced), while continuing to always run static ast-grep rules.
Changes:
- Add runtime rule discovery, narrowing (ast-grep), isolated
check.tsinvocation via bundledtsx, andFinding→CheckResultmapping. - Rewrite
taskless checkdispatch to always run static rules, gate runtime rules by auth/reconcile (or--dangerously-run-scripts), and expose skipped runtime rules in--json. - Add unit/integration tests and update CLI help/docs to reflect the new runtime rule model.
Reviewed changes
Copilot reviewed 17 out of 18 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| pnpm-lock.yaml | Locks the new tsx dependency version used to run runtime checks. |
| packages/cli/package.json | Adds tsx as a runtime dependency for executing check.ts. |
| packages/cli/src/types/runtime-rule.ts | Introduces the runtime rule contract types (Match, Finding, etc.). |
| packages/cli/src/schemas/check.ts | Extends --json output schema with optional skipped runtime rules. |
| packages/cli/src/rules/runtime/discover.ts | Discovers runtime rule directories and loads capture rule metadata. |
| packages/cli/src/rules/runtime/narrow.ts | Runs the runtime rule “narrow” via ast-grep and normalizes matches. |
| packages/cli/src/rules/runtime/invoke.ts | Executes check.ts under tsx with timeout + error isolation. |
| packages/cli/src/rules/runtime/harness.ts | Orchestrates narrow→gate→invoke and maps findings to CheckResult. |
| packages/cli/src/rules/runtime/run-set.ts | Signs/reports check.ts only and materializes blessed runtime rules into .run/. |
| packages/cli/src/commands/check.ts | Rewrites check to always scan static rules and conditionally run runtime rules, surfacing skipped runtime rules in --json. |
| packages/cli/src/help/check.txt | Documents runtime rules, modes, --dangerously-run-scripts, --timeout, and skipped JSON output. |
| packages/cli/src/help/ci.txt | Updates CI guidance: static rules always run; runtime rules require auth/verification. |
| packages/cli/test/runtime-harness.test.ts | Adds unit-ish tests for discovery/gating/mapping/timeout behavior. |
| packages/cli/test/runtime-check.test.ts | Adds integration tests for check mode table and reconcile interactions. |
| packages/cli/test/run-set.test.ts | Removes obsolete tests for the deleted static run-set gating logic. |
| packages/cli/test/reconcile-check.test.ts | Removes obsolete reconciliation behavior tests superseded by runtime gating behavior. |
| openspec/changes/runtime-rule-execution/tasks.md | Marks the OpenSpec task groups as completed and records implementation notes. |
Files not reviewed (1)
- pnpm-lock.yaml: Generated file
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This was referenced Jul 3, 2026
…ntation Verification (opsx:verify) caught a spec-vs-impl divergence: the narrow requirement, proposal, and design named `--inline-rules --json=stream`, but the harness assembles the capture rules into a temp `--config` (--inline-rules carries only one rule; a runtime rule has multiple capture rules + full ast-grep config). Reworded to "one scan per mode" via a generated config so the spec that gets synced to canonical on archive matches reality. Refs TSKL-245 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sync the runtime-rule-execution delta specs into the canonical specs and archive the change (verify → sync → archive). - cli-runtime-rule-execution: new canonical spec (the runtime harness contract). - cli-check: static-vs-runtime dispatch, the validated-path rule, skip+report, and --dangerously-run-scripts added; auth-state/reconcile/degrade requirements updated for the cutover; the now-obsolete 'warns on reconciliation mismatches' and 'exits cleanly when the run set is empty' requirements removed (their static-reconcile behavior was deleted in the cutover — the delta records the removal with reasons). - cli-rule-reconciliation: reporting + run-set requirements rescoped to each runtime rule's check.ts. Change archived to openspec/changes/archive/2026-07-03-runtime-rule-execution/; no unarchived changes remain, so the tip's check-openspec-archived job passes. Refs TSKL-245 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Six fixes from Copilot's review of #50, plus the canonical-spec twin of the #49 doc fix: - invoke.ts: on Windows, kill the check via taskkill /T (negative PIDs aren't supported), so a timeout actually terminates the tsx+node tree. - discover.ts: pin the check file to check.ts inside the rule dir; do not resolve metadata.taskless.check as a path (prevents escaping the dir via ../). - run-set.ts: signRuntimeChecks is now per-rule resilient (returns unreadable rules instead of throwing); reported reconcile paths are POSIX-normalized so Windows backslashes don't defeat the server-side path match. - check.ts: a missing/unreadable check.ts is reported as skipped and materialization errors degrade to a runtime-skip — a malformed runtime rule no longer aborts the whole check (static keeps running). - narrow.ts: run one broad scan per broad capture rule so matches are attributed to the right rule (was mislabeling all broad matches as the first rule); treat a signal-killed narrow (null exit code) as a failure instead of success. - specs/cli-runtime-rule-execution: fix the materialize requirement's live-tree path (.taskless/rules/ -> .taskless/runtime-rules/) to match #49. Add an integration test: a runtime rule missing check.ts is skipped (not fatal) and static rules still run. Full suite green (339). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Propagate the #49 review-comment fixes into the canonical specs (the archived change copy came via merge): - cli-check: degrade --json scenario allows the additive skipped array; --dangerously-run-scripts described as 'without server validation'. - cli-rule-reconciliation: heading renamed to 'Reconcile reports every runtime rule's check.ts'. - cli-runtime-rule-execution: per-capture match mode; broad matches are path-only (line/column 1, empty text/captures). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
thecodedrift
added a commit
that referenced
this pull request
Jul 6, 2026
Six fixes from Copilot's review of #50, plus the canonical-spec twin of the #49 doc fix: - invoke.ts: on Windows, kill the check via taskkill /T (negative PIDs aren't supported), so a timeout actually terminates the tsx+node tree. - discover.ts: pin the check file to check.ts inside the rule dir; do not resolve metadata.taskless.check as a path (prevents escaping the dir via ../). - run-set.ts: signRuntimeChecks is now per-rule resilient (returns unreadable rules instead of throwing); reported reconcile paths are POSIX-normalized so Windows backslashes don't defeat the server-side path match. - check.ts: a missing/unreadable check.ts is reported as skipped and materialization errors degrade to a runtime-skip — a malformed runtime rule no longer aborts the whole check (static keeps running). - narrow.ts: run one broad scan per broad capture rule so matches are attributed to the right rule (was mislabeling all broad matches as the first rule); treat a signal-killed narrow (null exit code) as a failure instead of success. - specs/cli-runtime-rule-execution: fix the materialize requirement's live-tree path (.taskless/rules/ -> .taskless/runtime-rules/) to match #49. Add an integration test: a runtime rule missing check.ts is skipped (not fatal) and static rules still run. Full suite green (339). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the
runtime-rule-executionOpenSpec change — the local harness that evaluates a runtime rule intaskless check.Stacked on #49 (
jakob/runtime-rule-execution, the proposal) → base this against that branch, notmain.All task groups complete (per the repo's
/opsx:applyconvention — commit + pause between groups):discover.ts+ contract types (types/runtime-rule.ts)narrow.ts(one ast-grep scan/mode),invoke.ts(check.ts viatsx, timeout + throw isolation),harness.ts(Finding→CheckResult)runtime/run-set.ts: sign check.ts only, per-rule eligibility, materialize blessed into.run/check.tsrewrite: static always runs, runtime gated by the mode table,--dangerously-run-scripts+--timeout,--jsonskipped. Cutover: removed static-reconcile gating (deletedrun-set.ts).check.txt(topic v2) +ci.txtruntime-harness.test.ts+runtime-check.test.tsModel: static ast-grep rules are trusted/inert and always run; a runtime rule's
check.tsis arbitrary code and runs only when server-verified (authed reconcile) or under--dangerously-run-scripts. Only thecheck.tsis signed/reconciled.typecheck+lint+ fulltestgreen (338 tests). Also fixed two real bugs found while testing: aFinding→CheckResultoff-by-one, and a timeout that killed only thetsxwrapper (now SIGKILLs the process group).Refs TSKL-245
Stack (root → tip):